Fixed-Weight Difference Target Propagation

Target Propagation (TP) is a biologically more plausible algorithm than the error backpropagation (BP) to train deep networks, and improving practicality of TP an open issue. methods require feedforward feedback networks form layer-wise autoencoders for propagating target values generated at output layer. However, this causes certain drawbacks; e.g., careful hyperparameter tuning required synchronize training, frequent updates path are usually that path. Learning sufficient make capable but having these necessary condition work? We answer question by presenting Fixed-Weight Difference (FW-DTP) keeps weights constant during training. confirmed simple method, which naturally resolves abovementioned problems TP, can still deliver informative hidden layers given task; indeed, FW-DTP consistently achieves higher test performance baseline, (DTP), on four classification datasets. also present novel propagation architecture explains exact function DTP analyze FW-DTP. Our code available https://github.com/TatsukichiShibuya/Fixed-Weight-Difference-Target-Propagation.